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ABSTRACT 


This  paper  points  out  the  application  of  dynamic 
programming  techniques  to  the  optimization  of  a  satellite 
reconnaissance  system  with  a  constrained  "picture-taking" 
capability.  The  technique  is  illustrated  by  an  example 
and  a  brief  description  of  a  possible  implementation  pro¬ 
cedure. 
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I.  INTRODUCTION 

A  satellite  reconnaissance  system  will  probably  have  a  restricted 
intelligence  gathering  capability.  This  restriction  may  be  due  to  limi¬ 
tations  on  the  data  which  can  be  transmitted  to  a  readout  station  during 
a  pass  or  the  film  available  in  the  satellite.  In  any  case,  there  is  a 
bound  on  the  "picture-taking"  capability  of  the  satellite. 

Generally  this  "picture-taking"  capability  will  be  a  parameter  in 
the  design  of  the  vehicle  and  the  over-all  command  and  control  network. 

We  may  select  this  parameter  by  deciding  on  the  maximum  (or  the  expected) 
number  of  targets  the  satellite  will  pass  over  between  readouts.  Gener¬ 
ally,  an  Increase  in  "picture-taking"  capability  will  decrease  the  resolu¬ 
tion  of  a  system  with  fixed  readout  capability  or  increase  the  cost  of 
the  system  by  demanding  a  greater  readout  capability  (i.e.,  larger  band¬ 
width  or  more  readout  stations) . 

As  some  targets  will  be  cloud  covered,  a  design  based  on  reserving 
"pictures"  for  all  targets  would  be  wasting  some  "pictures"  at  the  sacri¬ 
fice  of  either  resolution  or  cost.  Hence,  it  may  be  reasonable  to  con¬ 
sider  designing  the  system  so  that  the  number  of  "pictures"  is  less  than 
the  total  number  of  targets. 

However,  this  design  creates  an  additional  problem;  "What  targets 
does  the  satellite  photograph?"  When  the  satellite  passes  over  a  target 
it  observes  the  conditions  over  this  target  and  has  a  probabilistic 
notion  about  cloud  conditions  over  the  future  targets.  Hence,  the  satel¬ 
lite  must  make  decisions  sequentially.  This  paper  points  out  the  appli¬ 
cability  of  dynamic  programming  to  the  optimization  of  this  problem.  In 
other  words,  a  dynamic  programming  solution  will  provide  a  criterion  by 
which  the  satellite  can  decide  optimally  when  to  photograph  a  particular 
target. 

A  particular  case  which  can  be  solved  intuitively  is  when  photo¬ 
graphic  conditions  are  binary  (cloud,  no  cloud)  and  the  targets  are  equal 
valued.  Then  the  satellite  should  always  take  pictures  under  "no-cloud" 
conditions,  for  this  is  the  optimum  policy. 
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However,  atmospheric  phenomena  may  be  such  that  picture  quality 
can  have  several  possible  values  or  a  continum  of  values.  Also  the  tar¬ 
get  pictures  may  not  be  of  equal  importance.  Clearly,  the  satellite 
would  need  a  more  sophisticated  method  of  deciding  whether  to  take  a 
picture  of  the  rather  hazy  target  which  is  approaching. 

This  paper  presents  a  dynamic  programming  formulation  for  this 
latter  problem,  and  applies  the  model  to  a  small  example. 

II.  FORMULATION  OF  THE  PROBLEM 

Let  the  state  of  the  reconnaissance  vehicle  be  given  by  the  two 
dimensional  representation  S(i,J)  where 

i  “  number  of  targets  remaining  until  a  readout  station  becomes 
available 

J  •  number  of  remaining  pictures  which  the  reconnaissance  vehicle 
can  take 

The  initial  state  of  the  vehicle  is  given  by  S(n,  m) .  Hence, 
n  »  total  number  of  targets 

m  ■  total  number  of  pictures  which  the  reconnaissance  vehicle 
can  take 

In  accordance  with  the  discussion  of  Section  I,  we  let  n  >  m. 

It  may  be  more  important  to  get  pictures  of  some  targets  then 
others.  Hence,  we  consider  the  value  of  target  pictures  to  be  given 
by 


where 


t(l),  t(2)  ...  t (n) 

0  £  t(i)  £  1  for  1  s  i  s  n, 


These  values  are  pre-assigned  and  do  not  change  while  the  reconais- 
sance  vehicle  passes  between  readout  stations.  We  designate  the  targets 
and  pictures  as  follows: 
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If  "1"  targets  remain,  then  the  i'th  target  is  the  target  approach* 
ing  and  the  first  target  is  the  target  immediately  before  the  readout 
station. 

Picture  quality  is  affected  by  a  number  of  generally  uncontrol¬ 
lable  factors  --  atmospheric  conditions,  sunlight,  etc.  We  can  model 
this  affect  by  calling  the  picture  quality  of  the  i'th  target  a  random 
variable,  q^ ,  (0  S  q^  S  1)  and  define  p(q^  *■  x)  ■  p^(x)  as  the  proba¬ 
bility  density  function  for  picture  quality  of  the  i'th  target. 

In  our  example  we  will  consider  these  random  variables  to  be  in¬ 

dependent  and  identically  distributed.  We  can  define  the  picture  value 
v^  of  the  i'th  target: 

-If  a  picture  is  not  taken  of  the  i'th  target,  v^  ■  0 

-If  a  picture  is  taken  of  this  target,  v^  *  t^q^. 

The  decision  facing  the  reconnaissance  vehicle  at  i'th  target  can 
be  expressed  as  follows: 

Given  the  present  state  is  S(l,j)  and  the  picture  value  of  i'th 

target  from  direct  measurement  is  v^,  should  the  i'th  target  be 

photographed? 

Let  us  define  a  decision  criterion  d(i,J)  for  state  S(l,j)  such  that 

-If  d(i,j)  then  a  picture  will  be  taken  of  the  i'th  target 

-If  v^  <  d(i,J)  then  a  picture  will  not  be  taken. 

A  listing  of  d(i,j)  for  all  1  and  j  allows  decisions  to  be  made 
for  any  state  S(1,J)  on  the  basis  of  the  picture  value  v(i)  of  the  tar¬ 
get.  Such  a  listing  is  called  a  policy. 

For  example,  the  following  listing  of  d(i,j)  represents  a  policy 


d(i,J>  -  0 


for  all  i  and  j. 
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This  means  that  the  first  m  targets  would  be  photographed  because  their 
picture  value  it  0  by  definition.  However,  as  some  of  these  targets 
may  be  clouded  over  (v^  »  0)  it  is  unlikely  that  this  policy  would  be 
optimum. 

The  dynamic  programming  problem  is  to  determine  the  optimum  policy, 
that  is,  determine  the  values  of  d(i,J)  for  all  i  and  J  that  will  maxi¬ 
mize  the  total  picture  value  for  the  n  targets  (or  m  pictures). 

The  procedures  of  dynamic  programming  are  inspired  by  the  Principle 
of  Optimality.  "An  optimum  policy  has  the  property  that,  whatever  the 
initial  state  and  the  initial  decision,  the  remaining  decisions  must 
constitute  an  optimum  policy  with  regard  to  the  state  resulting  from 
the  first  decision." 

To  apply  this  principle,  we  must  define  V(i,j)  as  the  expected 
total  value  of  the  remaining  J  pictures  when  the  optimum  policy  is  fol¬ 
lowed  from  the  present  state,  S(i,j), 

We  can  then  write  the  basic  dynamic  programming  equation. 

Vi.J)  “  {pr(i,J)  [E(vt)  +  V(i-1,  J-l)]+  C  1-prCi, J)3  [v(i-l,J>]} 

where  pr(i,J)  is  the  probability  of  photographing  a  target  at  state 
S(i,J)  or  pr(i,J)  -  probability  [vt  £  d(i,j)] 

where  E(v^)is  the  expected  picture  value  of  the  i'th  target  under  the 
condition  that  a  picture  is  taken  of  this  target. 

Briefly,  this  relation,  Eq.  1,  means  that  V(i,j)  is  maximized  for 
any  state  S(i,J)  with  respect  to  the  decision  to  be  made  immediately, 
e.g.,  d(i,J).  Hence,  a  reverse  solution  is  obtained  from  V(0,0)  through 
successive  values  of  i  and  j  to  V(n,m). 

Now  applying  Eq.  1  to  this  particular  problem 


(1) 
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V(i,J)  -  (  I  tix  Mx>  dx  +  v(1_1»  J*1)  J  Pi.(x)  dx 


d(i,j) 


d(i,J) 

+  V(i-l.J)  J  p 

O 


t(x)  dx} 


(2) 


Taking  partial  derivative 


fdU ) j)'  "  pi^(l,J)]  -  V(irl.j-l)  PlCd(i,j)] 


+  V(i-l,J)  Pt[d(i,J)] 


and  setting  it  equal  to  zero 

d(if j)  -  (3) 

1 

This  is  a  rather  intuitive  result ;  it  says  we  take  a  picture  of  the 
i’th  target  if  the  value  of  this  target  is  greater  than  the  difference 
between  expected  value  of  remaining  i-1  targets  when  picture  ie  taken 
and  when  picture  is  not  taken. 

Mow  substitute  Eq.  3  into  Eq.  2, 

1  1 

V(i,J)  -  J‘  t.x  p  (x)  dx  +  V(i-l.J-l)  J  p  (x)  dx 
d(i> J)  1  d(l,J)  1 

d(l.J) 

+  V(1-1,J)  J  pt(x)  dx  (4) 


By  noting  the  initial  conditions 
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(a)  V(i,0)  ■  0  for  all  1,  and 

1 

(b)  V(l,l)  *  vx  *  J  xp1(x)  dx 

o 

we  can  solve  Eq.  4  repeatedly  for  Increasing  i  and  j.  Also  using  Eq.  3, 
we  can  determine  the  optimum  policy,  d(i,j). 

This  completes  the  dynamic  programming  solution.  An  example  in 
the  following  section  will  illustrate  the  procedure. 

III.  EXAMFU 

Assume  all  targets  are  equal  valued 

t.  «  1  1  £  i  £  n 

Assume  the  random  variables  for  picture  quality,  q^,  are  mutually  in¬ 
dependent  and  identically  distributed  random  variables  from  the  cumu¬ 
lative  distribution  below. 

probability  (q^  £  x)  *  0.3  +  0.7  x 

This  distribution  is  both  discrete  and  continuous  which  demands  the  use 
of  distribution  functions  rather  than  density  functions. 
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This  distribution  corresponds  to  a  situation  with  a  discrete  prob¬ 
ability  p(qt  •  0)  ■  0.3  for  cloud  cover  (q^  *  0)  and  a  continuous  uniform 
probability  for  other  improved  values  of  picture  quality. 

Let  us  assume  there  are  five  targets  and  three  pictures,  m  *  5, 

n  *■  3. 

The  V(1,J)  have  been  computed  from  Eq.  4  and  are  given  in  the  fol¬ 
lowing  matrix. 


S 

D 

2 

3 

■9 

0 

•  _ 

■1 

0 

-- 

-- 

mm 

0 

0.50 

0.70 

-- 

mm 

BE 

0.59 

0.93 

1.05 

mm 

B 

-- 

1.07 

1.32 

■1 

B 

-- 

1.52 

The  omitted  elements  correspond  to  states  which  the  reconnaissance  vehi¬ 
cle  can  not  attain. 

From  the  V(i,j)  matrix  the  d(i,J)  matrix  can  be  computed  by  using 

Eq.  3. 


\  J 
i  X 

0 

1 

2 

3 

0 
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• 

• 

l 

- 

0 

- 

- 

2 

- 

.35 

0 

- 

3 

- 

.50 

.20 

0 

4 

- 

- 

.34 

.12 

5 

“ 

“ 

- 

.25 
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To  illustrate  these  computations,  consider  a  specific  state  is  three  tar¬ 
gets  and  two  pictures  remaining,  S (3 , 2) ;  a  picture  should  be  taken  of  the 
third  last  target  if  its  picture  quality  is  greater  than  0.20,  (q^  S  0.2) 
(e.g.,  d(3,2)  *  0.20).  The  remaining  expected  picture  quality  (before 
condition  of  third  last  target  is  observed)  is  given  by  V(3,2)  «  0.93. 

Now  consider  an  alternate  policy  of  photographing  the  first  three 
targets  without  observing  the  photographic  conditions  which  is  the  ex- 

V(5 ,3)  -  3(0.35)  ■=  1.05 

pected  value  of  the  reconnaissance  mission  in  terms  of  picture  quality. 
This  compares  unfavorably  with  the  V(5,3)  ■  1.52  for  the  optimum  policy 
from  the  dynamic  programming  solution. 

Let  us  consider  another  policy  of  photographing  the  first  three 
targets  without  cloud  cover  (v^  /  0). 


prob 

(number  of 

clear 

targets 

E 

0) 

E 

0.0024 

prob 

(number 

of 

clear 

targets 

E 

1) 

E 

0.028 

prob 

(number 

of 

clear 

targets 

E 

2) 

m 

0.13 

prob 

(number  of 

clear 

targets 

E 

3) 

0.31 

prob 

(number 

of 

clear 

targets 

B 

4) 

E 

0.36 

prob 

(number 

of 

clear 

targets 

- 

5) 

- 

0.17 

1.00 


.'.  Expected  value  of  reconnaissance  mission 

-  (0.0024)  (0)  +  (0.03)  (0.5)  +  (0.13)  (1.00)  +  (0.84)  (1.50) 

-  1.41 


For  obvious  reasons,  this  policy  is  better  than  the  previous  one, 
however,  dynamic  programming  yields  an  even  greater  expected  value. 
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The  difference  which  depends  on  the  shape  of  the  picture  value 
probability  distribution  is  not  very  great  in  this  case.  For  other 
distributions  it  would  be  greater  or  less.  Some  general  study  of  the 
effect  of  this  distribution  could  be  performed. 

IV.  CONCLUSION 

The  purpose  of  this  paper  was  to  apply  dynamic  programming  to  a 
satellite  reconnaissance  problem  involving  sequential  decisions  concern¬ 
ing  when  observations  should  be  made.  The  paper  formulates  the  dynamic 
programming  model,  and  illustrates  its  use  for  a  particular  example. 

The  example  indicates  that  the  dynamic  programming  technique  provides 
substantial  improvement  in  reconnaissance  performance  over  less  sophis¬ 
ticated  decision  methods. 

In  this  example,  the  distribution  function  for  picture  quality  was 
selected  quite  arbitrarily.  In  a  design  study,  this  distribution  would 
be  obtained  from  actual  observations.  The  example  does  Indicate  that  it 
may  be  profitable  to  place  infrared  or  albido  measurement  devices  in  the 
satellite  so  that  picture  quality  can  be  determined  before  film  is  ex¬ 
posed. 

In  order  to  implement  the  dynamic  programming  technique,  the  re¬ 
connaissance  system  would  need  a  ground  capability  to  calculate  the 
decision  matrix,  d(i,J),  from  the  statistics  of  meterorological  condi¬ 
tions.  This  matrix  would  be  fed  into  the  satellite  before  passage  over 
targets.  The  satellite  would  compare  measurements  with  the  matrix  to 
arrive  at  reconnaissance  decisions. 

A  final  caveat  is  worth  noting,  the  motivation  for  this  study  and 
preceeding  discussion  has  been  the  assumption  that  the  picture  taking 
capability  will  be  a  design  or  operational  constraint.  If  this  constraint 
is  not  important  in  practice,  then  much  of  this  discussion  is  inappro¬ 
priate. 
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